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ABSTRACT 

The. effects of a school district's testing and 
evaluation activities cannot be effectively assessed without an 
understanding of %he explicit and implicit goajs embedded in the 
policies and practices of the district. A structured workshop process 
to elicit a district's intentions related to the effects—uses anq 
impacts — of testing and evaluation on instruction (T/E/l) involves 
participants from many levels of the school district organization. 
The process is meant to ascertain the implicit expectations of policy 
stakeholders within the organization a.nd those outside. The process 
can also be a planning' device to create a T/E/I linking system. The 
workshop allows diverse perspectives from many levels and ' 
incorporates procedures which build "ownership" in ideas or policies. 
A minimum of 8, and maximum of 16, people participate for several 
hours in procedures incorporated from Mason and Mitroff 's 
"ill-structured problems" methodology. The group generates and 
classifies, examples of effects of testing and evaluation using a 
brainstorming nominal group technique. In subgroups the participants 
determine the importance and certainty of effects, specify acceptable 
evidence of effects, select those effects to be measured, and develop 
instrumentation to measure effects. Pilot applications in "a 
simulation and. sample district illustrate the procedure. (CM) 
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• INTRODUCTION - 4 

Over the past two years we have intensely studied six school districts^ 

which were'identified as involved in activities [to .link their testing and/ 

or evaluatton activities with instruction (T, E,and I).. We . documented wH^t 

they were doing, how they came to be doing what they are doingi'hovl-.they 

were set up pperatibnal ly, and how they thought a]l of these efforts linked 

testing with instruction. From the outset of tha^ttudy ✓ we anticipated that « 

stage of the research which asks, in effect, "So* What! " 

^ The, "so what'^ question deals with the impact of Jfche testing and. evaluation^ 

activities of a given district. In the original project proposal , 

we expressed the issue, as that of understanding "the impact or effect of 

district-wide testing and evaluation activities on the actions of teachers 

and principals in classrooms and schools." (CSE Plan, 1979, p. 18.)* We 

wanted to assess the extent to which* the TEI linkage subsystem' was having, 

the "desired" or "expected" effects in'classrooms. 

V ' - 

. We have spent considerably time during the. past year examining the 

t 

topic of T/E impact assessment. Through dialogue, review of related work, 
"and some pilot applications we have -reformulated the Originaliissue, re- 
fined bur definitions of T/E impact, and outlined a procedure forfothers to 

use for themselves in clarifying their thinking about T/E impact'.- Tne pur- 
•pose of this pa'pV is to share our progress in working through these *funda- 
mental methodological issues. 
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Addressing the Issue . • 

Our original statement. of the issue, by focusing on "desired or ex- 
pected effects in. classrooms, 11 , suggested that we would look directly in 
classrooms for teacher behaviors which would indicate that they are select- 
ing and performing actipns basjra on input from testing and/or evaluation 



data. The first step we should take in designing our research, then, would 
be to spell out what we would look for as evidence of such effects. In 
order to do >this, we examined what districts were doing in testing and 
evaluation and attempted to develop reasonable scenarios -for what the com- 

mon impact might be. We askexJ ourselves: "Given this district's particular 

* i * m ■ 

testfng and evaluation subsystem, what types of- effects would flow from it 

and be evident in classroqms?" m 

The first thing we learned was that we were confused by the terms we 

had been taking as .synonymous , "Effects*^ "Impact," and "Use" are not 

terms .that we could continue to use interchangeably unless we, Wanted to 

stay hopelessly confused. -To clarify our .purposes, we adapted the defi- 

nitions suggested by Smith (1981) for the terms "use" and "impact." Smith 

defined "use" as "conscious employment of an evaluation (or test) to 

achieve! some desired end or impact," and "impact" -as "any discernible ac-. 

tions, events, or ; changes in conditions that are directly influenced by 

the evaluation (or testing activities) , 1 ts . processes , products or. find- * 

g ngs.'l E xtending these definitions , we add that uses aire intended /effects 

whereas impacts can be either intended or unintended . That is, examples 

of either "uses" or "impacts" can both be referred to as "effects" of a 

T/E/I linking subsystem. ^ 

Returning to the effort to specify indicators, we built a matrix as* in 

» v 

Table 1. < ' 
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TABLE 1: Specifying Effects 





Uses 


Examples , of 

Impaqts 


Sample Indicators 


Tests 


< 


Intended 


Unintended* 

. \ 


(a) All GATE candidates 
have 98% CTBS scores 


1 - CTBS 
2 

3. 


(a) Scores Iwifl be 
used tcL identify 
candidates for 
the GATE Program 


(c) Teachers identify 
potential candidates 
for GATE, notify 
special services. 


' \ 

\D) leacners use resuiys 
to assign reading • 
groups 


' (b) Examination of 
classroom reading 
group proii les snows 
.homogeneous grouping 
by CTBS scores.^ 


* * 

n * 

Evaluations 

1 

2 

• . 3 * 




s 

J 


* 


(c) Check in cum. cards 
indicates GATE 
children who were 
cited by teachers. 

. - ■ \ 


n 


• * 


/ 
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Using the datg from our field'work in six districts. to try out this 

matrix led us to the delineation of several key dimensions concerning 

the assessment of TE Use/Impact- They are listed herein random order. ' 

0 Testing and evaluation use ?md impact can both occur at many 
levels of the district's organization in addition to the 
classroom. 

0 .Testing and evaluation use and impact may be understood 
differently at different levels of the organization, e.g., 
administrators focus' on teit scores in relation to an 
\ entire program while teachfers see scores in relation to 

individual children. 
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'° Among the variety of observable impacts, some may be' in- 
tended by someone in a decision-making posrtion; some v 
may be unintended , * 

Many of the likely T/E effects are not recognized or arti- 
culated by members 9f the organization. 

° There may be a lack of consensus among members of the 
. organization 'over what constitutes "acceptable evidence" - 
'that effects are in fact' occurring. - " 

* . r * 

. ° Among the varieties of effefcts which can be described, some 
can be categorized as having a directly technical, that is, 
instructional, emphasis while others reflect a social/interpersonal 
emphasis which may or may not indirectly effect instruction. 

0 Among the' varieties of effects some are experienced inside 

the organization while others are felt in the environment \ 
oy±|ide the/organization, e.g., in the community, by the 
meSa, etc. ^ 

Neither anticipated use nor anticipated impact are typically 
built into the organization as clearly as one^would expect/ 

Assessing the intraprg^izational and thJ^environmental effects of 
testing and evaluation irv instruction requires amuch broader perspective 
-fhan that which ye/ had first anticipated. Not only must effects besought in 
places other than the classroom; not only must effects on instruction in- 
clude use as well as intended and unintended impacts; not only must 
the direct effect on instruction be accounted for, but also the indirect 

, ;. " ' a. 



effects mediated through the social/interpersonal processes, 

We must therefore step back to the level of school district ideas,, 
policies, and practices. Me need to find way to assess the effects of 
both the articulated policies and practices of the district related to 
the*use of testing and evaluation for instructional change as, well as the 
una^ticulated, intentions. Much of the confusion contributing to multi- 



level assessments of the effects of testing and evaluation stems from the 
fact that many district attitudes relied to testing and evaluation are 



not stated as a consistent policy position, but are evolutionary, reactive 

to circumstances (e„g., public oytcry at declining scores), or dependent > 

.-■<»■ 
on the preferences, of those in key opinion leader positions. 

Therefore, defining^ the effects that tesg^^ma^valuation are Sup- 
posed to have is not a simple matter of asking one or more policy makers 
or of searching for a written statement of district ipol icy. Ideas, ,pov 
licies, practices, and expectations change as they filter through the 
organization and through people's perceptions., The effects— that is, the 
uses and the impacts of testing and evaluation— occur differentially at 
different levels as thiVfil tering process occurg, ' ^ 

District intentions regarding testing and evaluation, then, are dy- 
namic in the sense tha,t x those who serve as transmitters of intentions are 
also adding to or. modifying the original intentions. This modification 
occurs at all levels of the organization and suggests that the transmit- 
ters or agents of intended, policy are also at all levels of thesystem-. To 

put it another way, ideas and policies are defined and implemented by all 

» 

those who have a stake in them. ..all such individuals can be called policy 

>.'■'. 

^stakeholders." ' 
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This stakeholder coqcept is of central importance' notion! y to those 
of us who would like to assess the effects of testing and evaluation on 
instruction but^also to those within a district who would lik£ to manage 
a testing/evaluation/instruction linkage. 

Testing and evaluation are activities which should be carried out with 
reference -tp (and deference to) stakeholders. 'Stak'ehplders are defined 
as "those^claimants inside and outside of the system -who have a vested in- 
terest irf'tffe problem under investigation and its* solution" (Mitroff & 
Mason, 1981>, Those working j^ith the stakeholder concept ask: Who is 
'affected; who has atfn interest-; who is in a position to effect adoption of 
results or execution of deci signs ; *who has ex}>ressed opinions;. who ought 
^tb^care about outcomes? The stakeholder concept is related to the pre- 
vious s'feat^ment that effects occur at'many levels and are defined differ- 
ently at different lWeJs, and further .complicated by the fact that effects 
occur both inside and /*^^ the system. Stakeholders, therefore, may 

be either ini^rnaf^WxteitS^to ttife school district organizatipn," 

To sunjmarize C^r^h^j^fg thus far:- the strandswhich come together 



are these/. 



' X . . . 

one cannot effectively assess the effects of & district s 

testing and evaluation activities without an understanding 

of the go^ls and intentions of" the district; 

district goals and intentions are embedded in tf^f ideas and 
in the policies and^practices of thjs district; 0 

some of the ideas and policies-are implicit rather/than 
explicit— unarticulateds rather, than artrculated; prac- 
tices may bi£ either consistent' or ir!fconsistent-wi th the 
prevailing* i<feas and policies; 

' \ 

the effort to make the ide*6 f and policies explicit must in- 
volve a wide range of stakeholders at all levels of the 



organization^ 
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t A PROCESS FOR USE/IMPACT (IarIfICATICJN 

""■ ' . ' <l 

- > / 

Over the past several months, our project staff has been developing 
*arid pilot testing a process 'to elicit a school district 'intentions re- 
lated to the effect of testing and evaluation on v if»struction. Thft pro- 
cessfuses a structured workshop in which participants from many levels of 
the school district organization collaborate. y 

In addition to enabling'a school distjrtct to ascertain the implicit T 
expectations of .stakeholders at many levels within the organization as well 
those outs\de the organization, it is our hope that the process can be. 



a planning device for districts seeking to create a T/E/I linking system. 

** \ ■ V - % ' 

The workshop format has two principal justifications. First, i£ allows 

participation of individuals from many levels whlj ha ve ; diverse perspec- 

tives, reflecting our belief that knowledge resides at many levels of the 

system. It is therefore not sufficient to explore use/impact intentions 

either for planning or for assessment with input from only the members of 

a testing ^nd evaluation unit. Second, workshops *can incorporate proce- 

dures which build "ownership" in ideas or policies.. 'We^hope that the' 



dynamic participative .workshop procedure that we are constructing meets 

*l these conditions. « • % 

Our wqrkshop requires*a minimum of 8, -a maximum of 16 people from 
across levels and functions in the district. It c^lls for at least 2h 



hours and is even more comfortable if conducted in a longer session. 

, The workshop procedures ,ctre adapt^i from Mason andj^jftroff^ (1981)* tffa 
who KS/e. applied their methodology for dealing with "i 11 -Structured . ' , 



problems"* in both public and private agencies* We have* incorporated 

their procedures into- five majot steps which, are first listed then dis- , * 

•- j 

cussed in detail. 
* ■ ■, * 

1. Generate examples of effects o'f testing and evaluation. 

2. Determine the importance and certainty of effects of 
testing and evaluation. * , 

3. Specify acceptable evidence of effects. 

4. Select those effects to be measured. 

5. Develop instrumentation to measure effects. 
1, Generate Examples of Effects 

The basic procedure used in this step is brainstorming -- a some- 
what structured form of brainstorming known as the nominal group technique. 
The nominal group technique requires that the group facilitator go from 
one .person to the next in turn, asking each person to contribute one or / 
more ideas to a group list which is being compiled. The us$ of nominal 
group process simply insures that each person in the group has an Oppor- 
tunity to contribute at least one item to the list. The process of going 
nominally from one participant to th? next is continued for successive 
rounds until members of the group begin to pass. When the point is reached 
that no other items are forthcoming- the process is terminated, 

The'lroup facilitator begins the process by asking for examples of 
some effects of testing and evaluation which participants have observed. 
No effort is made to focus or channel the items at this point. 

*I11 -structured problems are defined as those for which "there are no 
single rixjht answers; there is no consensus even on the definition of * ■ 
the problems; and .action steps will or should be taken in spite of 
these ambiguities" (Mason & Mitroff, 1981, p. 29); 



Furthermore, no effort is made to judge items contributed to the list. 

When it is clear that the brainstorming process has reached an N 
end, that is, when items are no longer coming forth from the group, the 
facilitator moves on to a sub-step to clarify what has happened in the 
group and possibly^timulate more thought by- clustering t^ items. A 
useful clustering strategy can be illustrated by the matrix in Figure 
1. We have found that examples generated by these brainstorming sessions 
can be roughly classified according to two dimensions. The \rst dimen- 
sion to be considered is the stakeholders to whom the examples apply and 
whether those stakeholders are internal or external to the school. The 
other dimension along which we find a great number of examples clustering 
is the emphasis of the effect. The emphasis of the effects tend to be 
either technical or social /interpersonal . 

When we take these two dimensions and overlay them to form a matrix, 
the resulting four quadrants combine different stakeholders with dif- 
' ferent effects. As an example of the use of the classification scheme., 
consider one of the examples from Table 1, "Teachers use results to assign 
reading groups." The stakeholders (classroom teachers and their students) 
are internal and the effect is technical, i.e., a technical type of in- 
structional decision. This effect ^as been placed in. the upper left 
quadrant. 
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Figure .1. Framework for Clustering 
• T/E Effects 

Technical Uses/Impacts 



Ex. 

Teachers use results ' 
to assign reading groups 



Internal 



O 



4-> 



Stakeholders to the 



o 

CO 
■r— 
tO 
(O 
jC 



Effect 



Social , Interpersonal 
Uses/Impacts 



\ 



External 



In pilot applications of the process we have drawn this figure 
on a blackboard or fVip-chart and illustrated to workshop participants 
how their examples can be grouped in one or the other of the quadrants. 
It is not necessary to classify every item on the diagram — a representative 
sampling is sufficient. The facilitator then asks the group if the frame- 
work suggests other examples of effects which could be added to the list. 
Oftentimes it will, and when^hose are added, the facilitator senses 
the time to move into step 2. 

• • ' r 



2. Determine the ImpbrtaRce and Certainty of Effects of Testing^and Evaluation 

In or^der to carry ou^this step the group is broken up into sub-groups. 
Sub-grouping is helpful because it provides an opportunity for more interaction 
and more contribution from group members. It also permits participants 
to consider a' selected sub-set of effects rather than the entire range. 

Tp subdivide the' group, participants are asked to select themselves, 
in fairly equal numbers, into one of t>he four quadrants. One group will 

then be considering the internal-technical effects, anothftr group will con- 
I ■ * 

centrate on the external technical, a third group will consider the inter- 

• • • * 

nal social-inter personal, a fourth will consider the external social inter- % 
personal. 

Once these sub-groups have been formed, the assignment for each group 
is to take all effects from the brainstorming phase which they feel jus- 
tified in placing in their quadrant and consider the importance/unimportance, 

certainty/ uncertainty of each of those effects in terms of the districts 

* * . *■ 

overall testing and evaluation effort. Once again, participants will be 

using a cross matrix, as illustrated in^igure 2. This time the horizontal 

line is scaled from important to unimportant while the vertical line is 

scaled from certain to uncertain. Each effect is discussed and placed on 

the classification scheme according to the consensus of the group. 

By way of example, consider a sample effect sueh as the following: 

test results used by remedial reading teacher to determine consonant blends 

to be reviewed. The group may agree that such use or impact of test results 

is important and place it far out on the Important dimension. They may 
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Figure 2. Determining Importance Certainty 



Certain 



Uniaportant 



j Um|p 



TP 



Important 

— a 



•Uncertain 



"Teacher use of test 
data to determine review 
needs , 
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disagree, however, on the certainly of % the effect. One person thinks * 
it is a very isolated occurrence; another saying that many teachers use 
the results in that way. The group decides to place the effect in the 
important/uncertain quadrant. IJ3 so doing they' have had "flagged" it as 
a potential topic for further study and have helped to clarify what it . 
means to them. | 

Once* the sub-groups have completed this- step they have, in essence, 
completed a first sorting of effect^ and made explicit their notions about 
which testing and evaluation effects are both "important and uncertain." 
Effects which fa11 int0 that quadrant of the classification scheme are 

• ' 17 . • 



regarded as the most critical effects according to the judgment of the 
particular small group. Each sub-groyp presents its classifications to 
the entire group. At this^point there may be discussion and some re- 
organizing Qf priorities. More important though,^jeach effect has been 

systematically considered. 

( 

3: Specify Acceptabl a Evidence Qf Effects 



4 Once again, participants work in sub-groups. This time each of the<- 
sub-groups is instructed to take those effects which it labeled as 
"important and uncertain '^Jmd produce for each an, example of evi- 
dence which might be useful ^in reducing the uncertainty about that 
particular effect. 

For the example used above; the group may decide that interview 
data indicating .that 2/3 of the remedial reading teachers used test results 
to plan their review activities would be acceptable evidence. -. 

When each of the small groups have completed this task, a general 
/group session is convened for each sub-group to present their T\st of 
effects and corresponding examples of evidence. 

The purpose of this s,tep is to involve school district personnel in 
the specification of data sets which they, themselves, will find acceptable 
The step is designed to help prevent us, as researchers, from designing and 
conducting a research study which can be summarily dismissed by its • 
intended clients. - > 

4. Select Effects to be Measured + 

Tfie effects and evidence lists from each group are compiled into 
one complete list. It is likely that tHfe composite list is too long 
for ^he time and Resources of most research. One viay to pare 'it down 



is\to have workshop participants rank order the total. list for research, 
priority. The ranking can be used to determine which effects should be- 

come the subject of continued research.. 

1 • • t ... > 

5. Develop Instrumentation, to Measure Effects .. 

This step requires, that the statements of evidence be used to develop 

instruments to collect data to prove the presence or absence, strength or 

weakness of selected effects. In our case. the instrumentation is designed 

by our group, the research team, and presented to the school district team 

for reaction and revision. Again, the intent is to involve the clients 

of the research in its design. * 

■ - . J 

The effects ' clarification process ends at this point. However, 

* * _/ 

the collaborati ve cl imate of the' process needs to continue ^through the 

implementation of the research. 

ii 

DISCUSSION OF PILOT APPLICATIONS' 
We have conducted two' trials of the clarification procedure. The 
first was a simulation using members of our Center staff in school district 
roles. This trial was devised to enable'us to try out, g revise, and refine 
the agenda.. The second was in one of the sample districts, Northtown, 
wherein we intended to proceed to #evel op 'instruments and assess 
X/E. effects. Recent events in the school district, however, precluded 
that opportunity and we proceeded only the point of -developing a set of 
research recommendations. ' 

A third trial in another of the sample districts was planned but 
again events in the district j(relevant environments) were such that.it 
could not be carried out. . ' 
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Staff Simulation ' • • <** 

Five* staff members assumed the roles of Director of a Testing and 
*Evaluation Unit, District Superintendent, principal , elementary teacher, 
and secondary teacher. AlV^bf the persons assuming roles had, at some 
time in their careers, w&ked for a public school "system -apd were familiar 
with school, issues . - » '. 

id order to set a context for the simulation,* participants read 
a sample district 'case study and assumed that scenario for their roles.V 

The' trial,- abbreviated by the fact that we had only 1- 1/2 hours 
in\*hich to conduct it, was carried out in our conference room. The 
experience suggested revisions to the^proce^s. These revisions will not 
be elaborated here. Instead, we present the content outcomes of the 
process. , . 

\ 
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^ , Table Z. Summary and Classification of Effects from 

Simulation Pilot 

Technical uses/impacts 



Helps teachers make better use of; 

' statistics 
"Builds curriculum -rigidity 
Enables teachers to speak a more common 

t • language ^ 
Promote^ contact with parents for home 

management ^ 
Kids are grouped 
' Meets individual i Instructional needs 

Tak£s up teaching t'irfle 
1 Counselors use to program sthidents^ 
1 Provides needs assessment information t5 



Internal 



Stakeholders ~° 



J*, 

i 



Kids get^grf^^ for sociometric 

pi^£&6$ 
Some kidsvv^t upset by tests 
Promo tes^con tact with parents on 

non-instractional as well as 

^instructional information 
Focuses irfca cognitive learning; ignores 

affective domain 
Intensifies competition between teach 

and between schools 



to . 



ers 



Provides ; achievement te^t 
Accountability function 
Enables school to receive funds 
Provjide^ reports to parents 



data for Board 



to effects 



Extern* T 



r » \ f 

Newspaper reports to community 

Gets greater/lesser public, support 

Consultation with parents 

Brings recognition from outside sources 



Social -interpersonal 
uses/impacts 
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Table 3 



: Listing of Important/Uncertain Effects and r 
Exalfies of Evidence from Simulation Pflot 



Important/Uncertain Effects 

Kids are grouped for sociometric 
•purpAses 



Focuses attention on cognitive over 
affective domain, 



. Bui 1 ds curri.cufl urn ri g,i di ty . 

i 

Enables schools to receive outside 
furrds. ^ 

Bri ngs . recogni ti on , to di s tri ct 
from outside sources 



Evidence 7 

Examine individual teacher grouping 
patterns; interview teachers for 
■rationale. * 

Observe classroom lessons; examine 
lesson plans; examine teacher 
questioning patterns* 

Interview teachers; examine teacher's 
objectives over %time* 

Interview Assistant Superintendent: 
interview funding sources about 
wh^ they funded district- 

* 4^ 

Interview educational colleagues, 
- applicants for positions in 
district. Interview 
researchers at university re 
district's reputation. 



All participants rank orderfed the effects listed in Table 3. 
Their rankings indicated that, were this an actual case rather than a 
simulation, the research would address the following issues, as priorities. 

1. Builds curriculum rigidity: (Does the testing ana evaluation 
activity have this effect?) 

2. Focuses attention on cognitive over affective domain: (Does 
the testing and evaluation activity have this effect?) 

r 



3. \Kids are grouped for sociometric purposes (Do teachers use 

T/E data in this way?) • ^ ■ * 

4. Enables school to receive outside funds (Have funds become . 
available because of J/E activity?) 

5. Brings recognition from outside sources (Can it be shown that 
the T/E activity has pojsitively increased the .district's visi- 
bility?) \ / 

* We point out that while numbers 1,2,3, are classrodm based effects", 

numbers 4 and 5 are the types of effects which cah have an indirect 

effect on classroom instruction. 

Because this was a simulation we did oot proceed to Step 5 - 

Develop instrumentation to measure effects. However, the results of 

the process prepare the research team for that step by r providing a client- 
s- 
centered focus.. r .' / 

Participants in the simulation,"* all experienced with general educational 
issues and with specific T/E issues, felt that the -process brought out 
aspects of testing and evaluation activities which theyjhad not considered. 
One of the participants described the process as "a series of sieves 
through which theossues get refined and focused." They felt the inter- 
action was particularly helpful to their new understandings. 

/ j, 
Pilot Application in North town. 

Through the Assistant Director of the RD and E Unit a two hour session 
was arranged in Northtown. Discussions with the Assistant Director pifior 
to. the workshop determined that the concentration during the workshop 



would be on the consolidated application -process. That' process, faefined 

in another project report (ED Project Annual Report, Nov 1980, p 82) ^ 

«. * •». 

can be* described as follows: * 

° Each consolidated-application school 's CTBS scores 

\ obtained initially are presented to each school's t , 

principal and staff along with the school 's mobility 
index, minority percent, and school enrollment 
figures « v \ 

° Based on these data, the schocfl staff, with the assis- 
tance of an Evaluation Services Office evaluator, de- 
termines a set of objectives and activities for the 
coming year. These form the core of the school's 
annual improvement plan. District evaluators regu- 
larly revisit these schools during the ensuing year. 
The CTBS tests are administered again in the Sprir^j and 
individual pupil results are reported to the appropriate 
teacher before the end of the school year. During 
the Summer, the Evaluation Services Office staff scores* 
the tests and analyzes the results in terms of the indi- 
vidual school 's stated goals. , "A school -specific report 
is prepared and presented to the school staff in the 
beginning of the FaTl quarter. This farms the basis 
for the school staff to reformulate* goafs and activities 
for the next year -- and the cycle is repeated. 

Eight school district representatives participated in the session: 
there were thtee from the RD & E unit, two from Ti^le I programs, 
one principal , and two resource teafchers. 

We opened the session with/a brief summary ^f the results of our 
research and an indication ef the next phase ~ assessing effects. It 
was clear from the start of the session, and in. fact it had been anti- 
cipated by' tfie Assistant Director pripmo the session, that there 

were many different agendas on the minds of participants. We also knew 

. t 

that the district was^in a state of anticipation of a potential court 
ruling on desegregation. The many unresolved issues and emotions 
absorbed some of the allotted time ancf the full process was not completed 
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Table 4: Summary and .Clustering of Effects. 

. the Consolidated Application Process 
in North town 



) 



Technical ,uses/impacts 



A 



° Repoyrt comes out too late to 
affect activities 

^CRT's aroused by teachers 

v ° Low CTBS scores led to strict 

classroom interruption rules 

to 

v 0 Process has no beatfng on what +J 
^ teachers do in^classrooms £ 

Internal ' - Stakeholders to 



o 



Certain minor! tyygrdjjps score 

poorly because ctf -language ^ 
problems \ J 1 

Principals set up in-service for 
PR reasons 

Testing takes up too much student 
time . 

Children are burned out from over 
testing ^ 



0 NRT results used by SEA 4 ^ 

0 RD & E staff conducts: .in-service 
when principal cannot, .Gets 
the RD & E.staff into some - 
schools t ■ 

0 CTBS analysis led to irk-service and 
* special materials i 

0 It satisfies reporting requirements 
the Effects ■ External 



Parents get a better view of what is 
. happening becatise of report 

Parents in many schools don't understand 
the report 

Public relations from report is good 
for some schools - bad for others 

I . 

Newspaper publishes NRT results 
Public is fixated on CTBS seores 



^ * Social -interpersonal >uses/ impacts 



ERLC 



Or 



Beyond *t|e generating of the-effects, the process ,got 
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bogged 



down "and tiji^had run out. When we reflected oh what , the session, told 



us about the effects of the .febnsoli dated application process, we con- 

J . ; < * ♦ , 

eluded that: 

0 the process is having its impact mainly in the external - 
technical and external /social -interpersonal 'area, but . 
very little impact oh the internal areas. It would-be 
wrong to even consider effects on classroom activity 
V because such effects have not been built into the. con- 
solidated application process j ( / 

°. the level and form ofNparticipation'^n the consolidated , N 
application process is5 differeniy#rom school to'schtfol 
and any effort to assess the process needs to use the, 
school site as the u(jit of analysis-; 

. & 

0 the "principal's behavior ^and attitude will bekey 
to level of use ij^ a given sqhool site: . * 

THese thoughts along with our suggestions for data collection procedures 
were shared with the .district personnels y 

■ v " 

- - • . v • 



Imp. 



V 



ERLC 
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